Non-Asymptotic Theory of Random Matrices Lecture 16: Invertibility of Gaussian Matrices and Compressible/Incompressible Vectors

We begin this lecture by asking why should an arbitrary n × n Gaussian matrix A be invertible? That is, does there exist a lower bound on the smallest singular value s n (A) = inf x∈S n−1 Ax 2 ≥ c √ n where c > 0 is an absolute constant. There are two reasons (or cases) which we will pursue in this lecture. 1. In Lecture 15 we saw that the invertibility of rectangular (i.e., non-square) Gaussian matrices yields invertibility of A for all sparse vectors. Specifically, we derived Sparse Lemma 5 which stated that There exists an absolute constant δ ∈ (0, 1) such that with probability 1 − e −cn inf x∈S n−1 (δn)-sparse Ax 2 ≥ c √ n. (1) 2. Suppose that the rows X 1 ,. .. , X n of A are " very " linearly independent. This has a geometric interpretation as we saw the Distance Lemma of Lecture 14. Let the hyperplane H k = span(X j) j =k. Then P(dist(X k , H k) < ε) ∼ ε and it follows that Ax 2 ≥ max k |x k | · dist(X k , H k) (2) for all x ∈ S n−1. This yields invertibility of A for all spread vectors, e.g., if |x k | ∼ 1 √ n , then Ax 2 ≥ 1 √ n · const.